feat: add TripAdvisor username scan module#428
Merged
kaifcodec merged 7 commits intoJul 23, 2026
Conversation
kaifcodec
added
the
need-extra-changes
Capture the member name, about text and avatar from OpenGraph tags via Result.taken(extra=...) when a profile is found.
Contributor
Author
|
|
TripAdvisor sits behind DataDome, which fingerprints the TLS handshake and returns 403 to Python's default HTTP stack regardless of headers. Route the module through a warmed, browser-impersonating curl_cffi session: a one-time homepage warm-up obtains the clearance cookie, and Chrome TLS impersonation clears the fingerprint check. Add reusable impersonate_validate/impersonate_request core helpers so other bot-walled modules can reuse the same warmed session, and fetch profile metadata (name, bio, hometown, joined, website, avatar) from TripAdvisor's persisted GraphQL query. Parsing is null-safe and fully guarded so metadata enrichment never turns a found account into an error.
Drop the leftover `re` import in tripadvisor.py (the website decoder no longer uses a regex). curl_cffi is fully typed but was not installed in the lint job. Install it there alongside httpx so mypy checks against its real types, and fix the two genuine mismatches instead of masking them: narrow the request method to Literal["GET", "POST"] and waive only the browser-name Literal on the impersonate argument.
brunolm
force-pushed
the
feat/add-tripadvisor-username-module
branch
from
July 22, 2026 12:22
3aa7310 to
e741806
Compare
This was referenced Jul 22, 2026
Owner
|
@brunolm we had recently introduced -t and -C flag in PR #416 One small thing I just noticed regarding timeout handling. The new Could you update both global_timeout = get_global_timeout()
timeout = global_timeout if global_timeout is not None else DEFAULT_TIMEOUTThat way, modules such as SpankBang, TripAdvisor, and Pornhub will automatically honor the |
impersonate_request() and the session warm-up used a hardcoded DEFAULT_TIMEOUT, so they ignored the CLI -t flag. Route both through a _timeout() helper that reads get_global_timeout() and falls back to DEFAULT_TIMEOUT, matching make_request() / generic_validate().
Contributor
Author
|
@kaifcodec that should be addressed now. I've also tested |
kaifcodec
added
the
user module addition
kaifcodec
approved these changes
Jul 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a username-existence scan module for TripAdvisor, plus a small reusable core helper for sites behind TLS-fingerprint bot walls.
How it works
GET https://www.tripadvisor.com/Profile/{username}to determine existence./Profile/...(TripAdvisor case-canonicalizes handles, e.g.travelbug→/Profile/TRAVELBUG, so a redirect to a profile URL still confirms the account exists).Why the new dependency (
curl_cffi)Since this module was first written, TripAdvisor moved behind DataDome, which fingerprints the TLS handshake (JA3/JA4) rather than the User-Agent. Python's default TLS stack (used by
httpx) is rejected with403no matter which headers or cookies you send — so the original plain-httpxapproach no longer works.The fix needs two things together:
curl_cffiwithimpersonate="chrome"— replicates a real Chrome TLS handshake so the fingerprint check passes.Neither alone is enough:
This lives in a new reusable core helper —
user_scanner/core/impersonate.py(impersonate_validate/impersonate_request) — that mirrorsgeneric_validatebut routes through a warmed, cookie-persistent impersonating session. Any other module that hits a DataDome/Cloudflare TLS wall can reuse it.curl_cffi>=0.7,<1is added topyproject.tomlandrequirements.txt.Testing
brunolm7,globetrotter,john,maria,david,susan) → Found with correct metadata; well-formed nonexistent handles → Not Found.tests/test_impersonate.pycovers warm-up-once/session-reuse and error handling.*_unreadabletests fail, and only on Windows).Context
This fills a real coverage gap: Maigret and WhatsMyName check TripAdvisor usernames but user-scanner did not.